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LONG-TERM  GOALS 

The  long-term  goal  of  this  work  is  to  examine  the  utility  of  commercial  bathymetric  lidar  technology 
solely,  and  in  combination  with  commercial  passive  imaging  spectrometers,  for  measuring 
environmental  information  for  military  applications  in  the  littoral  zone.  These  findings  will  indicate 
how  commercial  systems  might  evolve  to  achieve  improved  performance  for  rapid  environmental 
assessment,  and  for  deployment  in  unmanned  aerial  vehicles. 

OBJECTIVES 

1 .  Develop  new  classifiers  using  rule-based,  blob-level  techniques  to  combine  lidar  and 
hyperspectral  data,  and  compare  new  results  to  those  achieved  using  pixel-level  fusion 
algorithms  developed  in  previous  phases  of  the  project. 

2.  Apply  a  new  inversion  algorithm  to  hyperspectral  data  to  produce  seafloor  refiectace  data 
cubes,  and  analyze  the  data  cubes  to  identify  minimal  spectral  bands  for  seafloor  classification. 

3.  Refine  the  water  column  volume  visualizer  developed  in  earlier  work,  so  that  it  may  be  used 
for  editing  and  analysis  of  lidar  data. 

4.  Improve  the  REA  software  making  it  more  robust  and  stable,  and  improve  the  user 
documentation. 

5.  Investigate  use  of  REA  software  with  data  produced  by  other  sensors. 

APPROACH 

In  earlier  work,  we  developed  techniques  to  accomplish  seafloor  classification  from  SHOALS  lidar 
data  alone,  and  by  fusing  SHOALS  data  with  CASI  hyperspectral  data  [1],  [2].  These  pixel-level 
fusion  algorithms  were  implemented  in  the  Rapid  Environmental  Assessment  (REA)  Processor,  and 
that  software  was  delivered  to  ONR  in  March  of  2008.  In  this  phase,  we  develop  more  sophisticated 
approaches  for  fusion  and  classification  by  increasing  the  level  of  abstraction  in  both  the  spatial  and 
information  contexts  of  the  measured  data.  Our  approach  includes  the  addition  of  a  mean-shift 
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algorithm  to  REA  for  generation  of  spatial  blobs,  and  the  application  of  set  theoretic  techniques  to  find 
the  intersection  and  union  of  blobs  identified  separately  in  the  lidar  and  spectral  data.  We  also  add  to 
REA  the  capability  to  produce  blob-level  seafloor  classifications  using  decision  rules  automatically 
generated  with  the  See5  software.  These  new  capabilities  support  the  analysis  of  the  extended 
information  feature  space  derived  from  both  the  active  and  passive  data,  with  the  goal  of  finding  a 
subset  of  optimal  features  for  seafloor  classification.  Subsequent  to  our  earlier  reported  results,  we 
developed  a  more  sophisticated  constrained  inversion  algorithm  to  produce  spectral  reflectance  images 
of  the  seafloor  [3].  In  this  phase  of  CEEIBS  we  use  those  algorithms  to  produce  improved  images  from 
the  raw  airborne  data  acquired  in  the  previous  phases. 

In  addition  to  our  efforts  to  improve  the  accuracy  of  seafloor  classification,  we  extend  the  functionality 
of  the  water  column  volume  visualizer  by  adding  the  capability  to  auto-detect  the  sea  surface  and 
seafloor,  and  develop  the  necessary  hooks  into  the  raw  data  to  allow  use  of  the  volume  visualizer  for 
data  editing. 

WORK  COMPLETED 

Funding  for  this  phase  of  CEUBS  was  awarded  in  February  2010.  In  the  subsequent  7  months  we 
completed  the  following  work: 

We  developed  a  blob  generation  procedure  based  on  a  mean-shift  algorithm  [4],  and  implemented  this 
blob  generator  in  REA.  Given  any  arbitraty  raster  spatial  image  stack  defining  a  feature  space,  our 
mean-shift  approach  searches  for  similarity  in  the  feature  domain,  and  connectivity  in  the  spatial 
domain,  to  segment  the  spatial  image  into  labeled  blobs.  This  blobbing  procedure  is  an  improvement 
over  simple  region-growing  algorithms  tested  earlier,  and  is  an  important  step  towards  implementing 
higher-level  fusion  algorithms  to  combine  the  lidar  and  hyperspectral  data. 

We  developed  and  implemented  into  REA  a  Bhattacharyya  Distance  (BD)  classifier  [5],  which 
measures  the  distance  between  the  data  distribution  of  a  blob,  and  the  data  distributions  of  ground  truth 
regions  of  interest  (ROI),  and  assigns  the  blob  to  the  nearest  ROI  distribution  in  terms  of  the  distance 
computed  within  the  feature  space. 

We  integrated  the  automatic  rule  generation  software,  See5,  into  REA  and  refined  REA  to  parse  the 
output  rules  from  the  See5  classifier  to  generate  rule-based  classification  images. 

Using  a  new  spectral  optimization  algorithm  computing  a  constrained  inversion  of  the  CASI  data,  and 
new  algorithms  for  estimation  of  SHOALS  backscattering,  we  re-processed  the  SHOALS  and  CASI 
datasets  to  produce  improved  seafloor  reflectance  images  from  the  active  and  passive  data.  All  new 
classification  work  is  based  on  these  images. 

We  implemented  algorithms  to  auto-detect  the  surface  and  seafloor  in  the  water  column  volume 
visualizer,  and  implemented  the  ability  to  rapidly  generate  water  column  visualizations  in  arbitrarily 
drawn  regions  of  interest. 

RESULTS 

We  implemented  a  mean-shift  blob  generator  into  REA  and  are  tuning  its  performance  for  improved 
sensitivity  and  accuracy.  We  are  also  using  it  to  understand  the  information  content  within  derived 
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feature  spaees.  For  example,  in  Figure  1  (b),  we  show  the  spatial  blobs  resulting  from  analysis  of  the  8 
information  features  generated  by  gray  level  co-occurance  matrix  (GLCM)  analysis  of  the  SHOALS 
seafloor  reflectance  image  of  Looe  Key,  Florida  (Figure  1  (a)).  The  8  features  generated  in  GLCM 
analysis  are;  mean,  variance,  homogeneity,  contrast,  dissimilarity,  entropy,  second  moment,  and 
correlation  [1].  In  blobbing  this  feature  space  we  used  L2-norm  as  the  similarity  metric.  Here,  our 
blobbing  algorithm  identified  seven  distinct  clusters  within  the  manifold,  and  labeled  similar  blobs 
with  the  same  color.  On  inspection,  we  see  that  two  different  sand  types  are  well-separated.  But,  due  to 
subtle  differences  between  the  GLCM  textures  for  sea  grass  and  the  linear  reef  in  the  top  third  of  the 
image,  those  two  types  are  not  well-separated  in  the  blob  image.  We  expect  this  separability  to 
improve  with  subsequent  tuning  of  the  mean  shift  parameters,  and  with  selection  of  other  information 
features  derived  from  the  lidar  measurements  (e.g.  rugosity  and  waveform  shape  parameters). 


(a)  (b) 


Figure  1.  Grayscale  Looe  Key  SHOALS  seafloor  reflectance  image  (a),  Blobbed  image 
using  all  the  eight  features  of  the  GLCM  analysis  (b).  Seven  different  texture  classes  are 

found  by  the  mean  shift  clustering  process. 

We  can  identify  blobs  in  the  CASI  seafloor  reflectance  datacube  using  the  same  mean  shift  blobbing 
procedure,  but  in  this  case  we  use  spectral  angle  mapper  (SAM)  as  the  similarity  metric.  In  Figure  2, 
we  show  results  using  the  first  12  spectral  channels  of  CASI  data  (wavelengths  less  than  600nm), 
where  the  CASI  seafloor  reflectance  images  have  been  inverted  using  the  Spectral  Optimization 
Algorithm  [3].  Here,  our  mean  shift  blobbing  algorithm  has  identified  4  modes  in  the  manifold 
corresponding  to  4  bottom  types  in  the  spatial  data.  Upon  comparison  with  Figure  1,  we  see  a  good 
separation  between  the  linear  reef  and  the  sea  grass.  This  success  arises  from  the  fact  that  the  water  is 
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very  shallow  in  this  dataset,  and  at  the  time  of  data  collection  it  was  very  clear.  Under  these  conditions, 
the  spectrometer  can  be  used  to  achieve  accurate  seafloor  classification. 


Figure  2.  RGB  composite  of  (Red:597nm  Green:537nm  Blue:477nm)  Looe  Key  CASI 
bottom  image  (a),  Blob  image  using  the  first  twelve  bands  (wavelength  <  600nm)  as  features 

of  the  CASI  bottom  image  (b) 

To  illustrate  the  difference  between  pixel-level  and  blob-level  classification  with  passive  seafloor 
reflectance  spectra,  we  apply  a  Bhattacharyya  Distance  (BD)  [5]  classifier  to  the  blob  image  shown  in 
Figure  2(b),  using  7  training  ROFs  belonging  to  four  different  sea  floor  bottom  types.  This  procedure 
creats  the  blob-level  classification  map  shown  in  Figure  3(a).  In  Figure  3(b),  we  show  a  pixel-level 
classification  produced  by  applying  a  maximum  likelyhood  classifier  (MFC)  to  the  same  initial  12 
channels  of  spectral  data,  using  the  same  training  set.  The  mapped  classifications  are  sand,  reef,  reef 
rubble,  and  sea  grass.  On  inspection,  the  two  classifications  look  similar.  But  a  formal  accuracy 
assessment  reveals  that  the  blob-level  classifier  is  better.  The  overall  accuracy  of  the  blob-level 
approach  is  82%,  whereas  the  accuracy  of  the  pixel  level  classifier  is  75%.  (We  show  the  complete 
accuracy  assessments  in  Table  1  and  Table  2). 
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Figure  3.  Classification  image  output  from  a  BD  classifier  applied  on  a  blobbed  image 
using  the  first  twelve  bands  (wavelength  <  600nm)  as  features  of  the  CASI  bottom 
image  (b).  Classification  image  output  from  a  Maximum  likelihood  classifier(MLC)  applied 
on  the  first  twelve  bands  of  the  CASI  bottom  image  (Sea  Grass:  green,  Coral  Reef:  dark  green, 

Reef  Rubble:  orchid,  and  Sand:  yellow) 
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Table  1:  Confusion  matrix  and  the  overall  accuracy  table  for  the  classification  image 

shown  in  Figure  3(a). 


Table  2:  Confusion  matrix  and  the  overall  accuracy  table  for  the  classification  image 

shown  in  Figure  3  (b). 


Class  Reef1  Reef  Reef  2  Sand  Sea  Sea  User 

Name  (Linear)  Rubble  (Deep)  1  Grassi  Grass2  Accuracy 


Reef1 

(Linear) 

1260 

203 

724 

Reef 

Rubble 

128 

2739 

387 

Reef  2 
(Deep) 

50 

88 

1249 

Sand 

1 

0 

0 

0 

Sea 

Grass  1 

3 

35 

1 

Sea 

Grass  2 

4 

2 

7 

Producer 

Accuracy 

87.2% 

89.31% 

53.7% 

76 

16 

17 

54.8% 

1 

33 

58 

81 .86% 

11 

11 

8 

88.1% 

3970 

0 

0 

100% 

0 

3123 

871 

77.4% 

0 

1283 

1073 

45.29% 

87.9% 

69.9% 

52.9% 

75.0% 
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We  believe  it  is  possible  to  improve  the  results  of  fusion-based  elassifieation  by  interseeting  spatial 
blobs  from  the  active  and  passive  data,  and  using  decision-level  strategies  to  classify  the  intersected 
blobs.  To  support  this  work,  we  integrated  the  See5  decision  rule  generator  into  REA.  It  is  our  intent  to 
use  these  tools  to  generate  a  general  set  of  decision  rules  applicable  in  a  wide  range  of 
geomorphologies  and  environments.  To  do  this,  we  must  process  a  large  number  of  datasets  from 
around  the  globe.  Unfortunately,  our  anticipated  access  to  the  Naval  Oceanographic  Office  (NAVO) 
data  library  was  not  granted  because  of  data  security  concerns  expressed  in  that  organization.  We  are  in 
the  process  of  obtaining  unclassified  U.S.  coastal  data  from  the  U.S.  Army  to  support  this  work  task. 

In  earlier  phases  of  CLUBS,  we  developed  a  novel  tool  to  visualize  lidar  data  in  terms  of  the  returned 
backscatter.  At  present,  this  tool  exists  as  a  only  visualization  tool  -  not  as  a  metric  tool.  In  this  phase 
of  CLUBS,  we  propose  it  to  expand  its  utility  for  data  editing. 

In  this  phase  of  work  we  developed  the  ability  to  draw  Regions  of  Interest  (ROIs)  under  which  the 
water  column  is  displayed.  This  is  a  significant  improvement  over  earlier  versions  in  that  the  gridding 
to  voxelize  the  water  column  occurs  only  after  an  ROI  is  drawn  by  the  user.  We  are  also  currently 
exploring  acceleration  strategies  including  moving  the  required  computations  onto  the  Graphical 
Processing  Unit  (GPU). 


Figure  4.  Volume  Visualizer  showing  the  data  cube  of  size  ~0.6x0.2  km^.  The  second, 
third  and  fourth  quadrants  show  the  top  and  lateral  slices  of  the  data  cube  (a)  and  Detected 

sea  floor  overlaid  as  a  vector  (b). 

We  have  also  added  built-in  routines  to  detect  the  water  surface  and  seafloor  topography  by  the 
implementation  of  edge  detectors  in  the  three  orthogonal  planes  defining  the  lidar  backscatter  data 
cube.  To  date,  we  have  used  Sobel  and  Laplacian  of  Gaussian  (LOG)  edge  detectors  with  good 
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success.  The  user  can  overlay  these  edges  as  veetors  in  eaeh  of  the  three  multi-planar  displays.  This 
funetionality  is  available  through  the  pull  down  menu  of  the  Volume  Visualizer.  We  show  preliminary 
output  of  this  funetionality  developed  in  the  reporting  period  in  Figure  4  (b). 

IMPACT/APPLICATIONS 

The  blob-level  elassifieation  proeedure  developed  in  CLUBS  shows  a  promissing  way  of  moving 
pixel-level  fusion  to  blob-level  fusion  for  Ildar  and  hyperspeetral  data.  We  have  learned  that  the  kernel 
density  estimation  and  mean  shift  algorithm  generate  impressively  well-defined  blobs  with  proper 
input  parameters.  To  determine  the  identity  of  the  blobbed  objeets,  we  also  have  to  develop  elassifiers 
for  blobs  or  adopt  deeision  making  algorithms,  like  the  rule-based  approaeh. 

The  Bhattaeharyya  Distanee  (BD)  elassifier  measures  the  distanee  between  the  data  distribution 
eorresponding  to  one  blob,  and  distributions  for  ground  truth  ROIs.  Our  results  indieate  that  higher 
aeeuraey  is  aehieved  following  this  approaeh  as  eompared  to  pixel-level  elassifiers.  But  the  BD  based 
elassifer  still  requires  a  training  sets.  We  are  developing  methods  to  use  automatie  rule  generation 
software  to  avoid  the  requirement  for  site  speeifie  ground  truth. 

The  faster  speed  of  the  volume  visualizer  makes  it  possible  to  use  this  3 -dimensional  visualization  tool 
to  understand  baekseattering  ehanges  in  water  eolumn  from  water  surfaee  to  the  seafloor.  It  ean  now 
readily  be  used  to  loeate  small  objeets  in  the  water  eolumn,  and  objeets  or  layers  floating  below  the 
water  surfaee.  This  enables  developing  automatie  3  direetional  seanning  eapability  to  deteet  outliers 
and  determine  depths  from  surfaee  and  seafloor  edge  boundaries. 

All  algorithms  and  software  have  been  integrated  into  the  Rapid  Environmental  Assessment  (REA) 
Proeessor. 

TRANSITIONS 

We  will  eommereialize  the  REA  proeessor  and  promulgate  its  use  for  a  wide  range  of  benthie  mapping 
applieations.  Eor  the  past  year,  the  Joint  Airborne  Lidar  Bathymetrie  Teehnieal  Center  of  Expertise 
(JAEBTCX)  has  served  as  a  beta  test  site  for  this  software. 

The  basie  funtionality  of  the  algorithms  will  be  adopted  into  the  Coastal  Zone  Mapping  and  Imaging 
Eidar  (CZMIE)  to  be  built  by  Opteeh  at  its  offiees  in  Kiln,  Mississippi. 

RELATED  PROJECTS 

Coastal  Zone  Mapping  and  Imaging  Eidar  (CZMIE).  CZMIE  is  a  strategie  partnership  between  Opteeh 
International  and  the  Department  of  Marine  Seienee  at  the  University  of  Southern  Mississippi  leading 
to  the  design  and  eonstruetion  of  a  next  generation  bathymetrie  lidar  to  improve  performanee  in 
shallow  water,  and  aehieve  water  eolumn  and  seafloor  eharaeterizations.  The  CZMIE  projeet  will  also 
establish  an  industry/government/aeademie  eenter  of  expertise  for  bathymetrie  lidar. 
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